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ABSTRACT 


We  consider  a  sensor  that  is  subject  to  false  positive  and  false  negative  errors.  The 
sensor  searches  for  stationary  threat  objects  such  as  ballistic  missile  launchers  or 
improvised  explosive  devices.  The  objects  are  located  in  a  certain  area  of  interest,  which 
is  divided  into  area-cells.  The  area-cells  are  defined  such  that  each  of  them  may  contain, 
at  most,  one  threat  object.  The  task  of  the  sensor  is  to  determine  if  an  area-cell  contains  a 
threat  object,  and  the  objective  of  the  searcher  is  to  maximize  the  number  of  correctly 
determined  area-cells. 

Since  definitive  identification  of  a  threat  object,  and  subsequent  handling  of  that 
threat,  are  done  by  a  limited  number  of  available  ground  combat  units,  the  correct 
determination  of  an  area-cell  is  crucial  for  better  allocating  and  directing  these  scarce 
resources.  We  develop  an  algorithm,  rooted  in  large  deviations  theory  and  stochastic 
approximation  theory,  that  leads  to  the  optimal  search  effort.  The  computed  allocation 
maximizes  the  expected  number  correctly  determined  area-cells  as  the  number  of 
available  looks  for  searching  becomes  large. 
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EXECUTIVE  SUMMARY 


Advances  in  sensing,  unmanned  aerial  vehicles  (UAVs),  and  satellite  technologies 
have  vastly  increased  the  military  use  of  aerial  or  space  sensors  for  detecting  threat 
objects  such  as  improvised  explosive  devices  or  missile  launchers.  These  advanced 
technologies  may  generate  powerful  and  effective  sensors,  which  necessitate  operational 
concepts  in  order  to  facilitate  their  efficient  utilization.  A  typical  scenario  where  such 
operational  concepts  are  needed  is  related  to  maritime  interdiction  missions  where  a  naval 
task  force  is  patrolling  a  certain  area  of  interest  (e.g.,  Horn  of  Africa)  attempting  to  detect 
and  intercept  threat  objects  (boats)  such  as  terrorists,  pirates  and  weapon  smugglers.  Such 
threat  objects  are  relatively  rare  and  therefore  it  is  safe  to  assume  that  if  the  area  of 
interest  is  divided  into  a  grid  of  area-cells  of  size,  say,  five  square  miles,  then  each  area¬ 
cell  may  contain  at  most  one  stationary  threat  object.  The  task  force  employs  unmanned 
aerial  vehicles,  equipped  with  electro-optical  or  other  sensor  for  surveillance  and  search. 
Once  a  threat  object  is  detected,  the  task  force  dispatches  an  armed  inspection  unit  to 
investigate  the  threat.  Obviously,  a  false  positive  detection  results  in  unnecessary 
deployment  of  the  intercepting  unit,  while  a  false  negative  detection  may  result  in  severe 
operational  consequences  such  as  terrorist  or  pirate  attacks.  Arguably,  the  cost  of  a  false 
positive  detection  is  much  smaller  than  the  cost  of  false  negative  detection.  The  objective 
is  to  minimize  the  expected  total  cost. 

In  this  thesis,  we  address  operational  concepts  associated  with  employing  sensors 
in  persistent  search  missions  over  an  extended  search  area.  Specifically,  we  consider  the 
problem  of  efficiently  allocating  adaptive  sensors  across  a  search  area  of  interest.  The 
sensors  are  adaptive  in  the  sense  that  the  search  plan  is  not  set  in  advance,  but  rather  is 
updated  in  real  time  during  the  search  process  as  new  information  is  generated  by  the 
sensor. 

Assume  that  one  sensor  is  assigned  an  area  of  interest  to  search,  which  is 
partitioned  into  a  grid  of  m  area-cells.  The  sensor  operates  in  glimpses  or  looks.  A  look 
may  be  viewed  as  a  nominal  period  of  time  for  inspecting  a  certain  area-cell.  The  sensor 
may  spend  several  looks  (i.e.,  extended  inspection  time)  in  a  certain  area-cell.  Each  look 


xi 


generates  a  cue  or  signal:  detection  or  no-detection .  A  cue  may  be  correct  or  erroneous 
(false  positive  or  false  negative).  Suppose  that  the  sensor  has  l  looks  that  it  can  apply  to 
the  search,  and  these  looks  are  allocated  to  the  various  area-cells  dynamically  as  the 
search  mission  evolves. 

When  the  number  of  looks  available  t  is  small,  this  can  be  done  by  solving  a 
dynamic  program.  However,  due  to  the  curse  of  dimensionality  [11],  the  computational 
cost  of  solving  a  dynamic  program  grows  exponentially  in  t  m,  which  precludes  its  use 
when  the  number  of  looks  available  l  is  moderately  large.  This  thesis  is  aimed  at  the 
situation  when  t  is  large.  The  optimal  search  effort  allocation  is  determined  in  two 
stages:  First,  presuming  knowledge  of  the  presence/absence  of  a  threat  object  in  each 
area-cell,  use  large  deviations  theory  to  characterize  the  optimal  effort  allocation;  and 
second,  use  adaptive  ideas  to  generate  a  search  sequence  that  provably  converges  to  the 
optimal  effort  allocation  determined  in  the  first  stage. 

Large  deviations  theory  [4]  suggests  that  the  probability  of  incorrectly 
determining  an  area-cell  decays  roughly  exponentially  fast  with  the  number  of  looks. 
Proposition  1  asserts  that  the  expected  number  of  incorrectly  determined  area-cells 
decays  exponentially  fast,  with  a  rate  that  is  the  smallest  decay  rate  amongst  all  area- 
cells.  This  in  turn  can  be  used  to  obtain  the  optimal  fractional  allocations;  the  exact 
expression  is  shown  in  Proposition  2.  Unfortunately,  the  optimal  fractional  allocations 
require  knowledge  of  the  presence  or  absence  of  the  threat  objects,  which  is  what  we  are 
trying  to  determine.  However,  one  can  take  an  adaptive  approach,  motivated  by  the  large 
deviations  results,  which  guarantees  optimal  allocation  of  the  search  resources  when  the 
number  of  looks  available  is  large. 

To  the  best  of  our  knowledge,  this  is  the  first  thesis  that  describes  the  optimal 
effort  allocation  in  the  context  of  target  searching  along  multiple  area-cells  where  the 
sensor  has  false  positive  and  false  negative  errors,  and  that  provides  an  implemental 
adaptive  algorithm.  We  do  not  consider  travel  time  among  the  area-cells  because,  for  a 
large  number  of  looks,  its  affect  on  the  optimal  effort  allocations  is  not  significant.  This 
happens  because  the  sensor  could  stay  in  the  same  area-cell  for  a  large  number  of  looks, 
then  move  to  another  area-cell  and  remain  there  for  a  long  time,  and  so  on,  all  the  while 


satisfying  the  optimal  search  effort  allocation.  When  the  number  of  looks  is  relatively 
small  one  should  consider  resource  constraints;  see,  for  example,  [13]. 
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I.  INTRODUCTION 


A.  OPERATIONAL  MOTIVATION 

Advances  in  sensing,  unmanned  aerial  vehicles  (UAVs),  and  satellite  technologies 
have  vastly  increased  the  military  use  of  aerial  or  space  sensors  for  detecting  threat 
objects  such  as  improvised  explosive  devices  or  missile  launchers.  These  advanced 
technologies  may  generate  powerful  and  effective  sensors,  which  necessitate  operational 
concepts  in  order  to  facilitate  their  efficient  utilization.  A  typical  scenario  where  such 
operational  concepts  are  needed  is  related  to  maritime  interdiction  missions  where  a  naval 
task  force  is  patrolling  a  certain  area  of  interest  (e.g.,  Horn  of  Africa)  attempting  to  detect 
and  intercept  threats  objects  (boats)  such  as  terrorists,  pirates  and  weapon  smugglers. 
Such  threat  objects  are  relatively  rare  and  therefore  it  is  safe  to  assume  that  if  the  area  of 
interest  is  divided  into  a  grid  of  area-cells  of  size,  say  five  square  miles,  then  each  area¬ 
cell  may  contains  at  most  one  stationary  threat  object.  The  task  force  employs  unmanned 
aerial  vehicles,  equipped  with  electro-optical  or  other  sensor  for  surveillance  and  search. 
Once  a  threat  object  is  detected,  the  task  force  dispatches  an  armed  inspection  unit  to 
investigate  the  threat.  Obviously,  a  false  positive  detection  results  in  unnecessary 
deployment  of  the  intercepting  unit,  while  a  false  negative  detection  may  result  in  serve 
operational  consequences  such  as  terrorist  or  pirate  attacks.  Arguably,  the  cost  of  a  false 
positive  detection  is  much  smaller  than  the  cost  of  false  negative  detection.  The  objective 
is  to  minimize  the  expected  total  cost. 

In  this  thesis,  we  address  operational  concepts  associated  with  employing  sensors 
in  persistent  search  missions  over  an  extended  search  area.  Specifically,  we  consider  the 
problem  of  efficiently  allocating  adaptive  sensors  across  a  search  area  of  interest.  The 
sensors  are  adaptive  in  the  sense  that  the  search  plan  is  not  set  in  advance,  but  rather  is 
updated  in  real  time  during  the  search  process  as  new  information  is  generated  by  the 
sensor. 
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B.  BACKGROUND 

The  theory  of  optimal  search  has  a  history  of  principal  importance  in  military 
operations.  The  theory  has  fundamental  applications  to  anti-submarine  warfare,  counter¬ 
mine  warfare,  and  search  and  rescue  operations.  The  books  [6]  and  [15]  are  classical 
references  in  this  area;  with  [17]  a  valuable  recent  reference.  Search  problems  with 
discrete  time  and  space  of  the  type  addressed  in  this  thesis  are  not  new.  Optimal 
whereabouts  search,  where  we  seek  to  maximize  the  probability  of  determining  which 
box  contains  a  certain  object,  is  studied  in  [1]  and  [5].  Chew  [3]  considers  an  optimal 
search  with  a  stopping  rule  where  all  search  outcomes  are  independent,  conditional  on  the 
location  of  the  searched  object,  and  the  search  policy.  Wegener  [18]  investigates  a  search 
process  where  the  search  time  of  a  cell  depends  on  the  number  of  searches  so  far.  A 
minimum  cost  search  problem  is  discussed  in  [12],  where  only  one  search  mode  is 
considered  and  the  sensor  has  perfect  specificity.  The  paper  [14]  deals  with  discrete 
search  with  multiple  sensors  in  order  to  maximize  the  probability  of  successful  search  of 
a  single  threat  during  a  specified  time  period.  Other  discrete  search  problems  are  studied 
in  [2,  10,  and  19].  However,  all  of  the  aforementioned  references  assume  that  the  sensor 
has  perfect  specificity,  that  is,  there  are  no  false  positive  detections.  Our  models,  which 
are  related  to  [7],  relax  this  assumption.  The  results  we  obtain  in  this  thesis  rely  on  the 
large  deviations  theory  [4],  and  on  stochastic  approximation  theory  [8]. 

C.  INTRODUCTION 

Assume  that  one  sensor  is  assigned  an  area  of  interest  to  search,  which  is 
partitioned  into  a  grid  of  m  area-cells.  We  assume  that  the  area  of  interest  can  be 
partitioned  in  such  a  way  that  each-cell  i,  for  i  =  1,2,  ...,  m,  contains,  at  most,  one  threat 
object.  The  sensor  operates  in  glimpses  or  looks.  A  look  may  be  viewed  as  a  normal 
period  of  time  for  inspecting  a  certain  area-cell.  The  sensor  may  spend  several  looks  (i.e., 
extended  inspection  time)  in  a  certain  area-cell.  Each  look  generates  a  cue  or  signal: 
detection  or  no  detection.  A  cues  maybe  correct  or  erroneous  (false  positive  or  false 
negative).  Suppose  that  the  sensor  has  l  looks  that  it  can  apply  to  the  search,  and  these 
looks  are  allocated  to  the  various  area-cells  dynamically  as  the  mission  evolves. 
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Let  =  1  if  area-cell  i  contains  a  threat  object,  and  f  =  0  otherwise,  leading  to 
the  hypothesis  H,o  :  C,  =  I ,  and  H,\  :  L,  =  0.  We  suppose  that  there  is  some  initial 
intelligence  about  the  presence  of  threat  objects,  which  is  manifested  by  a  prior 
probability, 

m  =  =1), 

for  i  =  1,  ...,  m.  This  intelligence  comes  from  exogenous  sources  such  as  satellite 
imaging,  communication  interception  and  human  intelligence.  The  sensor  is  characterized 
by  its  sensitivity  and  specificity.  For  each  area-cell  i  we  have 

cij  =  Pi  (sensor  indicates  detection  in  area-cell  i\f  =1), 
which  is  called  the  sensitivity  of  the  sensor;  the  specificity  of  the  sensor  is  1  -  bj,  where 
bj  =  Polsensor  indicates  detection  in  area-cell  i\C,  =  0). 

Although  the  a/ s  and  P’s  may  depend  on  the  area-cell,  we  assume  that  they  do 
not  depend  on  the  number  of  looks.  Without  loss  of  generality,  we  take  >  bj ,  because 
we  can  reverse  the  cue  if  cij  <  b  i.  We  explicitly  assume  that  cij  f  bj,  for  otherwise,  the 
sensor  would  not  provide  any  valuable  information.  We  assume  that  the  collection  of 
search  results  from  the  various  looks  is  independent  for  a  given  area-cell  (meaning  that 
there  is  no  systematic  bias  in  the  sensor),  and  that  the  results  for  different  area-cells  are 
independent. 

D.  RESEARCH  MOTIVATION  AND  GOAL 

The  motivation  of  this  thesis  is  to  characterize  effort  allocation  schemes  that 
employ  the  sensor  efficiently,  where  the  measure  of  effectiveness  is  expected  number  of 
correctly  determined  area-cells.  When  the  number  of  looks  available,  l,  is  small,  this 
can  be  done  by  solving  a  dynamic  program.  However,  due  to  the  curse  of  dimensionality 
[1 1],  the  computational  cost  of  solving  a  dynamic  program  grows  exponentially  in  t  m, 
which  precludes  its  use  when  the  number  of  looks  available  l  is  moderately  large. 

This  thesis  is  aimed  at  the  situation  where  l  is  large.  The  optimal  search  effort 
allocation  is  determined  in  two  stages:  First,  presuming  knowledge  of  the  true  status  of 
state  of  nature  (i.e.,  equal  to  0  or  1,  for  i  =  1,  ...,  m),  use  large  deviations  theory  to 
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characterize  the  optimal  effort  allocation;  and  second,  use  adaptive  ideas  to  generate  a 
search  sequence  that  provably  converges  to  the  optimal  effort  allocation  determined  in 
the  first  stage.  To  the  best  of  our  knowledge,  this  is  the  first  thesis  that  describes  the 
optimal  effort  allocation  in  the  context  of  threat  searching  along  multiple  area-cells  where 
the  sensor  has  false  positive  and  false  negative  errors,  and  provides  an  implementable 
adaptive  algorithm.  We  do  not  consider  travel  time  among  the  area-cells  because,  for  a 
large  number  of  looks,  it  affects  on  the  optimal  effort  allocations  is  not  significant.  This 
happens  because  the  sensor  could  stay  in  the  same  area-cell  for  a  large  number  of  looks, 
then  move  to  another  area-cell  and  remain  there  for  a  long  time,  and  so  on,  all  the  while 
satisfying  the  optimal  search  effort  allocation.  When  the  number  of  looks  is  relatively 
small,  one  should  consider  resource  constraints;  see,  for  example,  [13]. 

The  remainder  of  this  thesis  is  organized  as  follows.  In  Chapter  II,  we  discuss 
what  we  mean  by  determining  an  area-cell,  using  results  from  the  hypotheses  testing 
literature.  In  Chapter  III,  we  determine  the  optimal  search  effort  allocations,  and  present 
an  adaptive  algorithm  that  results  in  sampling  allocations,  which  converge  almost  surely 
to  the  optimal  allocations.  In  Chapter  IV,  we  present  a  few  examples  that  illustrate  the 
main  results.  The  concluding  remarks  appear  in  the  last  chapter. 
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II.  PROBLEM  FRAMEWORK 


A  GOAL 

A  key  issue  is  the  determination  of  whether  an  area-cell  contains  a  stationary 
threat  object  or  not.  Intuitively,  the  average  number  of  detections  in  area-cell  i  should 
approach  a,  if  =  1  and  if  =  0.  Since  a,  >  bt,  a  good  decision  rule  should  conclude 
that  a  threat  is  present  if  the  average  number  of  detection  cues  in  that  area  is  close  to  or 
above  a,,  and  that  a  threat  is  not  present  if  the  average  is  close  to  or  below  Z?,.  Hence,  as 
each  area-cell  is  looked  at  more,  it  will  become  less  likely  to  have  a  wrong  determination. 

B.  SET-UP 

This  problem  has  been  well  studied  in  the  statistical  literature;  see,  for  example, 
[9],  In  this  section,  we  make  the  relevant  ideas  precise  and  set  the  stage  for  the  main 
results. 

Let  the  random  variable  XL]  =  1  if  the  j  ’th  look  into  area-cell  i  is  a  detection  signal, 
and  Xij  =  0  otherwise,  for  /  =  1,  . ..,//,  where  L  is  the  number  of  looks  into  area-cell  i. 
For  each  area-cell  i,  (Xij  :  j  =  is  a  collection  of  independent  and  identically 

distributed  random  variables  with  Bernoulli  (a,)  distribution  if  there  is  a  threat  object,  or 
Bernoulli  (b,)  otherwise. 

A  decision  test  S  is  a  sequence  of  measurable  maps  with  respect  to  Xn,  Xit 
from  {0,1}^'  into  {0,1}  such  that  if  S,  (xil,...,xi  f  )  =  1  then  Hiyo  is  accepted,  and  if 
Sf  (xiV...,xiJ  )  =  0  then  H/o  is  rejected.  The  error  probabilities  produced  by  the  decision 
test  S  are 

a,  =  P\(St  rejects  Hiyo),  (Type  I  error  probability) 

and 

P(  =  P\(  S,  accepts  Hi, o),  (Type  II  error  probability) 

We  define  the  sample  average  of  the  sensor  signals,  by 
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W,)=t'Lx,j, 

Ki  7=1 


and  let  S*  (x;.  j xt  t )  =  1  if 


f  _  \ 


v  h,  J 


1-a 


>1, 


(1) 


and  £)*  )  =  0  otherwise.  In  other  words,  S*t  )  =  1  if  and  only  if  the 

likelihood  of  having  a  threat  object  is  greater  than  the  likelihood  of  not  having  a  threat 
object. 


The  Bayes  probability  of  error  is  given  by  at  ni .  +  J3f  (1  -  ni .) .  Chemoffs  bound 
([4],  pp.93)  asserts  that  ifO  <Kt<  1  then 

.  .  .  1  .  , 


(!-*,■)]  =  - 

Yt  lo§ 

(  \ 
Zl 

+  (1-7;)  log 

( ,  Xl 

1  “7, 

yatj 

s  /,-» oo£.  ''  ' 

where  the  infimum  is  taken  over  all  the  decision  tests,  and 

iog| 


r\-h A 


7i  = 


log 


r  ai  x~bi^ 

v1  -at  bt  J 


Several  comments  are  in  order.  The  infimum  above  is  achieved  by  the  decision 
test  S *  so  that  S*  is  optimal  in  the  following  sense:  among  all  the  decision  tests,  it 
minimizes  the  Bayes  probability  of  error  in  log  scale  (i.e.,  it  is  the  decision  test  with  the 
largest  Bayes  probability  of  error  exponential  decay  rate),  as  the  number  of  looks  i(  — >  oo. 

Hence,  for  the  rest  of  this  thesis,  we  deal  with  the  decisions  test  S *  for  each  area-cell  i  = 

1,  ...,  m. 


By  a  straightforward  algebraic  manipulation,  it  can  be  seen  that  (1)  holds  if  and 
only  if  Xi(£j)  >  yr  Thus  the  parameters  yi  are  determination  thresholds,  meaning  that  if 
Xi(£i)  <  yt,  then  the  sensor  operator  declares  that  the  area-cell  does  not  contain  a  threat 
object,  and  if  Xi(£i)  >  yt  then  the  area-cell  is  declared  to  contain  a  threat  object. 
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III.  IMPLEMENTATION  OF  ALGORITHM 


A.  ALGORITHM  APPROACH 

Let  9(  be  the  fraction  of  the  search  budget  l  that  corresponds  to  area-cell  i,  so 
that  the  number  of  looks  into  area-cell  i  is  £i  =  9f£ .  To  make  the  mathematical 
proceedings  less  cumbersome,  we  work  with  Xi(9i£)  (the  average  of  the  observations 
taken  over  9/,  looks),  rather  than  with  Xi(\_9il J),  where  [-J  is  the  floor  operator.  Since 

our  results  hold  for  l  large,  they  continue  to  be  true  when  the  integrality  condition  is 
enforced,  by  working  with  a  sequence  that  goes  to  infinity. 

Assume,  without  loss  of  generality,  that  area-cells  1,...  ,  r  contain  a  threat  object, 
and  that  area  cells  r  +  1,  ...  ,  m  do  not  contain  a  threat  object.  Recall  that  the  goal  is  to 
allocate  the  sensor  in  order  to  minimize  the  expected  number  of  incorrectly  determined 
area-cell.  More  precisely,  the  goal  is  to 

min  gt{9{,...,9m), 

subject  to 

m 

i. 

i=i 

9>  0  for  i  =  1,  ...,  m, 

where 

r  m 

gM^9m) = <  r,] + £  nx.m  >  rj  •  (2) 

i=l  i=r+ 1 

Our  contribution  is  two-fold:  (i)  we  characterize  fractional  allocations  9* ,...,9*u 
that  are  optimal  (in  log  scale)  as  £—>  oo;  and  (ii),  we  provide  an  easily  implementable 
algorithm,  rooted  in  stochastic  approximation  theory  that  results  in  sampling  allocations 
that  provably  achieve  the  same  performance  (in  log  scale)  as  the  optimal  allocations  in 
the  limit  as  l  — >  oo. 

Large  deviations  theory  [4]  suggests  that  each  of  the  summands  in  (2)  decays 
roughly  decays  exponentially  fast  with  the  number  of  looks.  The  decay  rate  depends  on 
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the  large  deviations  rate  function  /,(/,)  =  sup7  {ijyi  -  log  E  exp^A., )} ,  which  for  a  non¬ 


degenerate  Bernoulli  random  variable  X^\  with  mean  //,■  is  given  by 


Ifri)  =  Yi  lo§ 

(  \ 
Yi_ 

+  (1-7;) 

(  i  b 

1  “  Yi 

kFu 

(3) 

for  0  <  y;  <  1;  see  Exercise  2.2.23  of  [4],  The  next  result,  whose  proof  can  be  found  in 
[16],  characterizes  the  decay  rate  of  the  expected  number  of  wrongly  determined  area- 
cells. 


1.  Proposition  1  -  Decay  Rate 

Suppose  cij  >  bifor  each  area-cell.  Then 

lim^log g,(0„...,0J  =  -  min  Of  ;(y.) .  (4) 

Proposition  1  asserts  that  the  expected  number  of  incorrectly  determined  area 
cells  decay  exponentially  fast,  with  a  rate  that  is  the  smallest  decay  rate  amongst  all  area- 
cells.  This  suggests  that  a  good  allocation  should  maximize  the  slowest  decay  rate,  i.e., 
the  minimum  of  0t  /  (;/  ) .  The  next  result,  from  [16],  shows  that  this  approach  is  optimal 
among  all  feasible  allocations. 


2.  Proposition  2  -  Optimal  Allocation 

Suppose  a,  >  bifor  each  area-cell.  Then 

lim— log  gA0f...,0*)  = - - - 


where 


0,  = 


i  x — T  m 


Er.,T(n) 


For  any  other  feasible  6m)  we  have 


lim  inf -log  )> - - - 


(5) 
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The  elements  0*  as  the  optimal  allocation  scheme,  meaning  that  no  other 
allocation  achieves  a  higher  exponential  decay  rate  for  g,  (•)  as  the  number  of  looks  goes 
to  infinity.  Proposition  2  results  in 

f  \ 

Yt  log  —  +(!-/,) log 

= _ _ 

u‘  r  f  \  ^r1’ 

Z*-1  ^log  Yk  +  (l~Yk) log  Yk 

L  Vvk)  U-/4JJ 

where  //,  =  a,-  for  i  =  1,  r,  and  fi,  =  bi  for  i  =  r  +  1,  m.  It  can  be  seen  from  (6)  that 
the  optimal  fractional  allocations  tend  to  be  large  when  //,  (i.e.,  a,  if  a  threat  object  is 
present,  or  bi  otherwise)  is  close  to  the  detennination  threshold  yi .  This  happens  because, 

in  that  case,  the  probability  of  having  X 1 1  on  “the  wrong  side”  is  relatively  large,  and 
more  looks  are  needed  to  compensate  for  the  bigger  error  probability. 

While  Proposition  2  characterizes  the  optimal  search  allocation,  the  fractions  9* 

depend  on  knowledge  about  the  presence  /  absence  of  the  threat  object,  which  is  precisely 
what  we  are  trying  to  determine.  In  the  next  section,  we  present  a  stochastic 
approximation  algorithm  that  overcomes  this  issue  and  leads  to  fractional  allocations  that 
almost  surely  converge  to  the  optimal  6]  allocations. 

B.  STOCHASTIC  APPROXIMATION  ALGORITHM 

First,  we  present  the  algorithm,  with  the  intuition  behind  it  following.  Initially  we 
set 

Ao  =  -T  ’ 

for  0  <  xt  <  1 ,  and 

f  \  f 

/,.o(T,)=T,l°g  +(!-/,) log  - — £~ 

KAi,0  J  ^;,o 

Based  on  prior  intelligence,  the  value  xt  is  our  best  guess  of  //,  at  stage  0;  for 
instance,  xt  =  a,  if  n>  0.5  and  xt  =  bi  if  nt  <  0.5.  The  rate  functions  that  lead  to  (6)  are 


9 


estimated  by  substituting  ji,  by  Xi0  in  (3).  The  initial  stage  is  l  =  0  and  the  initial 
sample  sizes  are  Xi0  =  0  . 


1.  Algorithm 

1.  Generate  a  replicate  2  from  the  probability  mass  function  V\  /  ^  '  1 1~\ , 


for  i  =  1, m. 


2.  Update  sample  sizes:  XftM  =  X^e  + 1 ,  and  Xi(+l  =  XtJ  for  i  • 

3.  Generate  a  sample  from  area-cell  £ ,  (say)  AT  x  . 


4.  Update  M  and  /s  M : 


X 


£,t+ 1 


=  Xt.+- 


X 


?,£+l 


and 


=  Yc  log 


v  ) 


+  (l-^)log 


V 1  Xgj+i  j 


For  i  ±  £  ,  set  Xi  M  =  XlJt ,  and  Ii  M  =  Ii  t . 

5 .  Increase  i  —>  i  +  1  and  go  to  step  1 . 

To  ensure  that  each  area-cell  is  searched  infinitely  often,  let  (vf  )J=0  be  an 
increasing  sequence  such  that  vf  — >  oo  and  J(vk  <f)  — >0,  where  J(-)  is  the 

indicator  function.  We  search  all  m  area-cells  at  iteration  v1,v2,...,  and  update  the 
parameters  according  to  step  2  and  4  of  the  algorithm. 

In  step  1,  we  decide  where  to  look  next.  This  is  accomplished  by  sampling  from 
the  probability  mass  function  /rj I~\,  which  is  the  best  guess  of  the  optimal 

allocation  at  stage  i  .  In  step  3,  the  searcher  generates  an  observation  by  sampling  from  a 
Bernoulli  distribution  with  parameter  a,  if  area-cell  2  contains  a  threat  object  (i.e.,  c 
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e  {1,  rj),  or  from  a  Bernoulli  distribution  with  parameter  bs  otherwise.  In  step  4,  we 
update  the  sample  average  and  sample  large  deviations  rate  function  of  area-cell  %  . 

The  see  why  our  algorithm  leads  to  the  optimal  allocations,  let  6n  =  Ailt  l  be  the 
fractional  allocation  in  stage  l  of  the  algorithm.  Hence,  step  2  of  the  algorithm  can  be 
expressed  as  0iM  =0i(  =i)-9i  f)/(T  +  l),  where  ^  is  the  £’ th  replicate  of  t, 

generated  in  step  1  of  the  algorithm.  The  recursion  for  9im  can  be  re-written  as 

Oi,i+ 1  =  ei,i  +  (&i  ~  &i,e  )  +  en 

where 

£>  =  J^J <6  =  0  -  (V.j:  -  °*)  > 

and  <7;.  f  =  77*  /  ^  " ,  l't  ■  If  the  error  s,  becomes  small  relative  to  (0*  -  9i ,)!  (t  + 1)  term, 
then  9i ,  follows,  as  l  —>  oo ,  the  path  of  the  solution  of  the  ordinary  differential  equations 

0\=0*-Gi, 

which  has  0*  as  the  unique  globally  asymptotically  stable  point.  This  suggests  that  if  the 
variability  introduced  by  the  error  is  sufficiently  small,  our  algorithm  provides  fractional 
allocations  that  converge  almost  surely  to  the  optimal  allocations.  The  preceding 
argument  leads  to: 

2.  Conjecture  1  -  Convergence  to  Optimal  Allocation 

The  stochastic  approximation  algorithm  search  allocations  converge  with 
probability  one  to  the  optimal  allocations  determined  by  (5); 

A-  f  * 

almost  surely  as  £  — >  oo. 

The  proof  of  this  conjecture  will  appear  elsewhere. 

Remark.  O*Ii(yi)  =  0*kIk(yk)  suggests  that  step  2  of  the  algorithm  could  be 
replaced  by  T  =  arg  min,  {AitIit  } . 
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The  preceding  can  leads  to  the  following  conjecture:  the  expected  number  of 
incorrectly  determined  systems  area-cells  produced  by  the  stochastic  approximation 
algorithm  approaches  0  at  the  best  possible  rate. 

3.  Conjecture  2  -  Expected  Number  of  Incorrectly  Determined  Area- 
cells  Approaches  Zero 

The  expected  number  of  incorrect  determinations  satisfy 

1  r  m  1 

7 1°8  I  s  /, )  +  I  >  r, )  -  „„  f ' . 

1  L~t  '-»!  J  (Tt) 

as  t  — >  oo. 
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IV.  NUMERICAL  VALIDATION 


A.  INPUT  PARAMETERS 

We  now  illustrate  the  results  of  the  previous  sections  via  three  examples  with  up 
to  12  area-cells.  In  the  first  case,  we  consider  4  area-cells,  with  just  the  first  2  area-cells 
containing  a  threat  object.  The  “baseline”  sensor  parameters  are  a  =  (0.8,  0.7,  0.6,  0.9), 
and  b  =  (0.3,  0.1,  0.2,  0.2).  These  parameters  lead  to  the  thresholds  y  =  (0.5609,  0.3608, 
0.3869,  0.5803)  and  to  the  optimal  allocations  (?  =  (0.2773,  0.1659,  0.4401,  0.1167). 


B. 


ILLUSTRATION  OF  PROPOSITION  1  AND  2 


In  the  first  example,  we  portray  the  results  of  propositions  1  and  2.  For  each  area¬ 
cell,  we  analytically  compute  the  exact  probability  of  incorrect  determination,  given  by 


z 


KJ 

k 


I  k  /"I  \i—k 

Vi  (’“«,) 


for  i  =  1 ,  and  by 


I 

kAlyfi'  1+1 


KJ 

k 


P-(i -b,y 


for  i  =  r  +  1,  ...,  m.  The  expected  number  of  incorrectly  determined  area-cells  is  the  sum 
of  these  two  expressions  over  all  the  area-cells.  This  is  shown  in  Figure  1,  for  three  cases: 


Optimal  allocation,  0*  (cf.(5)  and  (6)); 


•  Constant  allocation,  with  6*  =  H  m  ; 

•  Simple  allocation,  where  the  determination  thresholds  are  the  midpoints  between 
cii  and  bi,  (ai  -  bi )/  2 .  If  the  allocations  are  inversely  proportional  to  the  distance 

a;.  -  (a,.  -It)  /  2 ,  the  allocation  becomes  9i  =  £(ai  -bt)  ’  /  (ak  -bk )  . 


As  expected  from  Proposition  1,  the  logarithm  of  the  expected  number  of 
incorrect  determinations  for  the  three  types  of  allocation,  decays  linearly  as  the  number  of 
looks  increases.  Also,  in  agreement  with  Proposition  2,  the  slope  of  the  line 
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corresponding  to  the  optimal  allocations  is  more  negative  than  the  slope  of  the  line 
corresponding  to  the  constant  and  simple  allocations.  Another  take-away  from  Figure  1, 
which  we  also  observed  through  numerical  experimentation,  is  that  the  sub -exponential 
terms  appear  when  computing  the  expected  number  of  incorrectly  determined  area-cells 
become  unnoticeable  for  relatively  small  values  of  l ;  i.e.,  we  get  a  straight  line  in  Figure 
1  even  for  i  small.  This  suggests  that,  although  all  our  results  apply  in  the  limits  as  t  — > 
oo,  the  transient  dies  out  fairly  quickly. 


Figure  1 .  Exponential  decay  rate  of  expected  number  of  wrongly  determined  area-cells 

for  optimal  allocation,  constant  allocation  and  simple  allocation. 

In  Figure  2,  we  illustrate  the  effect  of  increasing  the  number  of  area-cells  for  the 
optimal  allocation,  and  of  having  the  difference  between  a  and  b  being  small. 
Specifically,  we  consider  the  case  a  =  (0.5,  0.7,  0.6,  0.4)  and  b  =  (0.4,  0.55,  0.55,  0.2). 

We  plot  the  case  of  4,  8  and  12  area-cells  where  the  parameters  and  number  of 
threat  objects  for  the  8  and  12  area-cells  scenario  are  copies  of  those  for  the  4  area-cell 
scenario.  As  can  be  seen,  the  performance  decreases  with  the  number  of  area-cells.  This 
can  be  justified  as  follows.  Recall  that  the  optimal  decay  rate  is 
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f  m  A-1 

I >:'(n)  ■ 

V  *=i  7 

Clearly,  the  above  expression  is  decreasing  in  m,  meaning  that  the  expected 
number  of  incorrectly  determined  area-cells  approaches  zero  at  a  slower  rate;  i.e.,  we  are 
bound  to  make  more  erroneous  determinations.  This  is  the  reason  why  the  slope  of  the 
lines  corresponding  to  m  =  8  and  m  =  12  becomes  closer  to  zero. 

We  observe  that  the  expected  number  of  incorrectly  determined  area-cells  is  much 
larger  with  the  current  a  and  b  than  with  the  baseline  parameters.  This  illustrates  the 
potential  operational  value  of  our  approach,  since  now  we  are  working  with  a  much 
larger  proportion  of  incorrectly  determined  area-cells. 


Figure  2.  Exponential  decay  rate  of  expected  number  of  wrongly  detennined  area-cells 

for  4,  8  and  12  area-cells  scenario  using  optimal  allocation. 

C.  ILLUSTRATION  OF  CONJECTURE  1 

In  the  last  example,  we  implement  the  stochastic  approximation  algorithm, 
without  presuming  knowledge  of  the  presence  or  absence  of  the  threat  object  in  each  cell. 
The  goal  is  to  illustrate  conjecture  1.  We  generate  a  single  replication  and  plot  the 
fractional  allocations  generated  by  the  adaptive  algorithm.  Figure  3  illustrates  how  the 
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Fractional  allocations 


allocations  approach  the  optimal  allocations  (6).  As  can  be  seen,  the  convergence  to  the 
optimal  allocation  is  already  fairly  good  for  100  looks,  and  becomes  even  better  as  the 
number  of  looks  increases. 


Figure  3.  Sample  allocations  versus  optimal  allocations. 
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y.  CONCLUSIONS 


A.  CONCLUDING  REMARKS  AND  FUTURE  WORKS 

In  this  thesis,  we  developed  a  model  for  a  single  sensor  that  searches  for  threat 
objects,  when  the  sensor  is  subject  to  false  positive  and  false  negative  errors.  The  space  is 
discrete,  with  each  area-cell  containing  at  most  one  stationary  threat  object.  We  employed 
the  large  deviations  theory  to  characterize  the  optimal  search  effort  allocations,  and 
developed  a  stochastic  approximation  algorithm.  The  fractions  of  time  spent  in  each  area¬ 
cell,  generated  by  the  algorithm,  converge  almost  surely  to  the  optimal  search  effort 
allocations.  This  causes  the  number  of  incorrectly  determined  area-cells  to  decay  at  the 
fastest  possible  exponential  rate.  Our  results  were  illustrated  via  3  numerical  examples, 
all  of  which  verified  our  conjectures.  Our  methodology  is  appropriate  when  the  number 
of  looks  is  large;  when  the  number  of  looks  is  small,  other  techniques,  such  as  numerical 
optimization  and  dynamic  programming  may  be  appropriate. 

The  models  developed  in  this  thesis  may  be  extended  in  several  directions: 
dealing  with  multiple  sensors,  consideration  of  an  arbitrary  number  of  targets  in  each 
area-cell  or  of  a  single  target  in  the  area  of  interest,  etc. 
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